Image processing method and camera thereof

ABSTRACT

An image processing method includes receiving image data and outputting an image stream including an initial group of pictures and a next group of pictures according to the image data. The initial group of pictures includes a first initial frame and at least one first non-initial frame. The next group of pictures includes a second initial frame and at least one second non-initial frame. The image processing method further includes obtaining a first ideal frame count of the next group of pictures according to a ratio relationship of frame sizes of the first initial frame and the at least one first non-initial frame of the initial group of pictures in a predetermined frame segment or a predetermined time segment and setting a frame count of the next group of pictures according to the first ideal frame count.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing method and a camerathereof, and more specifically, to an image processing method forobtaining an ideal frame count of a next group of pictures of an imagestream according to a ratio relationship of frame sizes of an initialframe and at least one non-initial frame of an initial group of picturesand a camera thereof.

2. Description of the Prior Art

In general, a camera device for image surveillance (e.g. IP camera)adopts a stream transmission method to immediately transmit capturedimages to a control terminal (e.g. a video control host) for real-timesurveillance. In practical application, for reducing the bandwidthoccupied by an image stream to save the transmission time of the imagestream, the image stream is usually divided into a plurality of groupsof pictures (GOP) and each group of pictures is composed of one intraframe and plural predicted frames (or bi-directional predicted frame).Since the predicted frame (or bi-directional predicted frame) only keepsa part of original image data, the aforesaid frame compression methodcould reduce the overall data size of the image stream for generatingthe image compression effect.

In the aforesaid image compression method, more frame count of thepredicted frame represents smaller data size of the image stream, whichmeans it is advantageous to reduce the transmission time of the imagestream. However, when the frame count of the predicted frame between thetwo intra frames is excessive, it may cause the problem that the qualityof the image stream is low, graphical glitch occurs easily when the datapackage for the image stream is lost during data transmission, and theplayback of the image stream is time-consuming since decoding softwarecould not find the intra frame quickly. Although the aforesaid problemcould be solved by reducing the frame count of the predicted frame (orbi-directional predicted frame) in each group of pictures, the overalldata size of the image stream could be increased accordingly in thismethod so as to cause the transmission delay problem.

On the other hand, if there are more moving objects in the imagescaptured by the camera device so as to cause greater variation in framesize, it is desired to make each group of pictures of the image streamhave a smaller frame count so that the image stream could have a higherquality and a faster image playback speed. If there are more stillobjects in the images captured by the camera device so as to cause lessvariation in frame size, it is desired to make each group of pictures ofthe image stream have a larger frame count so that the overall data sizeof the image stream could be reduced for saving the transmission time ofthe image stream.

In summary, how to optimize the frame count of each group of pictures inthe image stream according to the content of the image stream is onemajor concern of the camera device in image stream transmission.

SUMMARY OF THE INVENTION

The present invention provides an image processing method. The imageprocessing method includes receiving image data and outputting an imagestream according to the image data. The image stream includes an initialgroup of pictures and a next group of pictures. The initial group ofpictures includes a first initial frame and at least one firstnon-initial frame. The next group of pictures includes a second initialframe and at least one second non-initial frame. The image processingmethod further includes obtaining a first ideal frame count of the nextgroup of pictures according to a ratio relationship of frame sizes ofthe first initial frame and the at least one first non-initial frame ofthe initial group of pictures in a predetermined frame segment or apredetermined time segment and setting a frame count of the next groupof pictures according to the first ideal frame count.

The present invention further provides a camera. The camera includes animage capturing unit, a processing unit, and a transmitting unit. Theimage capturing unit is used for capturing images of an external scene.The processing unit is connected to the image capturing unit forreceiving image data and generating an image stream corresponding to theimages of the external scene according to the image data. The imagestream includes an initial group of pictures and a next group ofpictures. The initial group of pictures includes a first initial frameand at least one first non-initial frame. The next group of picturesincludes a second initial frame and at least one second non-initialframe. The processing unit is further used for obtaining a first idealframe count of the next group of pictures according to a ratiorelationship of frame sizes of the first initial frame and the at leastone first non-initial frame of the initial group of pictures in apredetermined frame segment or a predetermined time segment and settinga frame count of the next group of pictures according to the first idealframe count. The transmitting unit is connected to the processing unitfor transmitting the image stream.

These and other objectives of the present invention will no doubt becomeobvious to those of ordinary skill in the art after reading thefollowing detailed description of the preferred embodiment that isillustrated in the various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram of a camera according to anembodiment of the present invention.

FIG. 2 is a flowchart of an image processing method according to anembodiment of the present invention.

FIG. 3 is a diagram of stopping generating non-initial frames of a nextgroup of pictures and directly generating an initial frame andnon-initial frames of a further next group of pictures.

FIG. 4 is a diagram of generating the initial frame and the non-initialframes of the further next group of pictures after generating an initialframe and the non-initial frames of the next group of pictures.

FIG. 5 is a diagram of setting a frame count of the next group ofpictures according to an ideal frame count of the further next group ofpictures.

DETAILED DESCRIPTION

Please refer to FIG. 1, which is a functional block diagram of a camera10 according to an embodiment of the present invention. As shown in FIG.1, the camera 10 includes an image capturing unit 12, a processing unit14, and a transmitting unit 16. The image capturing unit 12 could bepreferably an image sensing module (e.g. image sensor and lens) forcapturing images of an external scene (e.g. parking lot). The processingunit 14 is connected to the image capturing unit 12 for generating animage stream corresponding to the images of the external scene andsetting a frame count of a next group of pictures in the image streamaccording to an expected compression efficiency and a ratio relationshipof frame sizes of an initial frame (e.g. intra frame) and at least onenon-initial frame (e.g. predicted frame or bi-directional predictedframe) of one group of pictures of the image stream in a predeterminedframe segment (e.g. in 30 frames) or a predetermined time segment (e.g.in 1s), so as to optimize the frame count of the next group of picturesof the image stream. The transmitting unit 16 is connected to theprocessing unit for transmitting the image stream in a wired or wirelesstransmission manner to a control terminal (e.g. a video control host),so as to achieve the real-time surveillance purpose.

The image processing method adopted by the camera 10 is described asfollows. Please refer to FIG. 1 and FIG. 2. FIG. 2 is a flowchart of theimage processing method according to an embodiment of the presentinvention. The image processing method of the present invention includesthe following steps.

Step 200: The image capturing unit 12 captures the images of theexternal scene.

Step 202: The processing unit 14 receives image data corresponding tothe images of the external scene and generates the image streamaccording to the image data.

Step 204: The processing unit 14 obtains an ideal frame count of a nextgroup of pictures according to the expected compression efficiency and aratio relationship of frame sizes of an initial frame and non-initialframes of an initial group of pictures in the predetermined framesegment.

Step 206: The processing unit 14 sets the ideal frame count as the framecount of the next group of pictures.

Step 208: The transmitting unit 16 performs transmission of the imagestream.

More detailed description for the aforesaid steps is provided asfollows. In Step 200, the camera 10 could turn on the image capturingunit 12 to capture the images of the external scene for subsequent imageprocessing of the processing unit 14. Subsequently, the processing unit14 could receive the image data for generating the image streamcorresponding to the images captured by the image capturing unit 12(Step 202), and could set the frame count of the next group of picturesof the image stream (Step 204). The image data could be uncoded originalimage data or coded image data and could be provided by an imagecapturing unit, a camera, the network, or other related storage device(but not limited thereto). If the image data received by the processingunit 14 is the coded image data, the processing unit 14 needs to decodethe image data into the uncoded original image data in advance beforegenerating the image stream. In this embodiment, the related descriptionfor Step 204 is provided as follows in the condition that the processingunit 14 sets the frame count of the next group of pictures (i.e. thesecond group of pictures of the image stream) according to the framesizes of the initial frame and the non-initial frames of the initialgroup of pictures (i.e. the first group of pictures of the imagestream). In Step 204, the processing unit 14 could obtain the idealframe count of the next group of pictures according to the expectedcompression efficiency (could be a system default value or a userdefined value) and the ratio relationship of the frame sizes of theinitial frame and the non-initial frames of the initial group ofpictures in the predetermined frame segment (e.g. in 30 frames, but notlimited thereto, meaning that it could be a predetermined time segment(e.g. in is) in another embodiment).

To be more specific, when there are more moving objects in the imagescaptured by the image capturing unit 12 so as to cause greater variationin frame size and a larger average frame size of the non-initial frames,the frame ratio of the frame size of the initial frame to the averageframe size of the non-initial frames becomes smaller, meaning that theframe ratio and variation in frame size are reversely related to eachother. Furthermore, as mentioned above, variation in frame size and theframe count of each group of pictures in the image stream could bepreferably in negative correlation. In summary, the frame ratio and theframe count of each group of pictures in the image stream could bepreferably in positive correlation. Thus, the processing unit 14 couldpreferably set the frame count of the next group of pictures of theimage stream according to the frame ratio of the frame size of theinitial frame to the average frame size of the non-initial frames of theinitial group of pictures. To be more specific, according to thepractical experience and the related formula derivation, the processingunit 14 could preferably obtain the ideal frame count of the next groupof pictures of the image stream according to the following formula.I _(GOP) =E(IP _(ratio)−1)/(1−E);

-   -   wherein I_(GOP) represents the ideal frame count, E represents        the expected compression efficiency (0<E<1), and IP_(ratio)        represents the frame ratio of the frame size S_(i) of the        initial frame to the average frame size S_(p) of the non-initial        frames of the initial group of pictures (i.e. S_(i)/S_(p)).

For example, in the condition that the frame rate (frame per second,FPS) of the image stream is 30 FPS, the expected compression efficiencyis set as 0.9, the initial group of pictures is composed of one intraframe and twenty-nine predicted frames, the frame ratio of the framesize S_(i) of the intra frame to the average frame size S_(p) of thetwenty-nine predicted frames is equal to 10, the processing unit 14could calculate the ideal frame count of the next group of pictures ofthe image stream as 81 according to the aforesaid formula, and thencould set the frame count of the next group of pictures as 81 (Step206). In such a manner, the transmitting unit 16 could transmit theinitial group of pictures and the next group of pictures as the initialgroup of pictures is composed of one intra frame and twenty-ninepredicted frames and the next group of pictures is composed of one intraframe and eighty predicted frames (Step 208), so as to optimize theframe count of the next group of the pictures in the image streamaccording to the content of the image stream. Thus, the presentinvention could achieve the purpose that the camera 10 could transmitthe image stream at the expected compression efficiency as the framecounts of the groups of pictures in the image stream are set to theideal frame counts corresponding to variation in frame size. As for therelated description for setting of the ideal frame count of othersubsequent group of pictures in the image stream (e.g. setting the idealframe count of the third group of pictures in the image stream accordingto frame sizes of an initial frame and at least one non-initial framesof the second group of pictures in the image stream), it could bereasoned by analogy according to the aforesaid embodiment and omittedherein.

To be noted, obtaining of the ideal frame count is not limited to theaforesaid embodiment. For example, in another embodiment, the presentinvention could adopt the design that the ideal frame count I_(gop) isdirectly obtained by looking up a predetermined table (preset by thepractical experience) of the frame ratio IP_(ratio) and the ideal framecount I_(gop) without the aforesaid calculation process, so as toenhance the calculation efficiency of the processing unit 14 inobtaining the ideal frame count I_(gop). The predetermined table of theframe ratio IP_(ratio) and the ideal frame count I_(gop) could be asshown in Table 1, but not limited thereto.

TABLE 1 IP_(ratio) I_(GOP) 0 ≤ IP_(ratio) < 5 30 5 ≤ IP_(ratio) < 8 60 8≤ IP_(ratio) < 11 90

Furthermore, for preventing frequent change of the ideal frame countI_(GOP) when the frame ratio IP_(ratio) varies at the boundary of theratio segment (e.g. the ideal frame count I_(GOP) varies between 30 and60 as the frame ratio IP_(ratio) varies between 4.9 and 5.1), theprocessing unit 14 could update the ideal frame count I_(GOP) only whendetermining the frame ratio IP_(ratio) continuously steps across morethan two levels of variation. For this reason, the present inventioncould further subdivide the ratio segments as shown in Table 1 into theratio segments as shown in Table 2 (but not limited thereto).

TABLE 2 IP_(ratio) I_(GOP) 0 ≤ IP_(ratio) < 3 15 3 ≤ IP_(ratio) < 5 30 5≤ IP_(ratio) < 7 45 7 ≤ IP_(ratio) < 8 60

That is to say, assuming that the processing unit 14 calculates thecurrent frame ratio IP_(ratio) as 4.9, the processing unit 14 couldobtain the ideal frame count I_(GOP) as 30 according to Table 2. Aftercalculation of the next thirty frames, assuming that the processing unit14 calculates the next frame ratio IP_(ratio) as 5.1, the processingunit 14 could obtain the ideal frame count I_(GOP) as 45 according toTable 2. At this time, since the ideal frame count I_(GOP) only stepsacross one level of variation, the processing unit 14 does not updatethe ideal frame count I_(GOP) meaning that the ideal frame count I_(GOP)is still set as 30. Subsequently, after calculation of further nextthirty frames, assuming that the processing unit 14 calculates thefurther next frame ratio IP_(ratio) as 7.1, the processing unit 14 couldobtain the ideal frame count I_(GOP) as 60 according to Table 2. At thistime, since the ideal frame count I_(GOP) has stepped across two levelsof variation, the processing unit 14 updates the ideal frame countI_(GOP) as 60.

Moreover, the processing unit 14 could further adopt the design that theideal frame count is directly set as a maximum value of the framesegment when the ideal frame count is within the frame segment (could bea system default segment or a user defined segment), for preventingfrequent change of the ideal frame count. For example, if the idealframe count of the third, fourth, fifth, and sixth groups of picturesare obtained as 10, 25, 35, and 55 respectively, the first frame segmentis 0˜30, and the second frame segment is 30˜60, the processing unit 14could set the frame counts of the third, fourth, fifth, and sixth groupsof pictures as 30, 30, 60, and 60 respectively.

For making update of the frame count of one group of pictures in theimage stream more smooth and further conform to the content of the imagestream in real time (e.g. the one group of pictures could have a lowframe count when variation in frame size is large, or could have a highframe count when variation in frame size is small), the processing unit14 could set the frame count of the one group of pictures according to acomparison result of the obtained ideal frame counts of the one group ofpictures and the next group of pictures. For example, if the ideal framecount of the second group of pictures (e.g. next group of pictures) ofthe image stream is equal to 60 and the ideal frame count of the thirdgroup of pictures (e.g. further next group of pictures) of the imagestream is less than 60 (e.g. 30), meaning that variation in frame sizeis gradually increasing (it represents that there are more movingobjects in the third group of pictures than the second group ofpictures), the processing unit 14 stops generating the non-initialframes for the second group of pictures and then directly generates theinitial frame and the non-initial frames for the third group ofpictures. That is to say, the processing unit 14 sets the frame count ofthe third group of pictures as 30 and directly generates one initialframe and twenty-nine non-initial frames for the third group of picturesafter stopping generating the non-initial frames for the second group ofpictures (as shown in FIG. 3). Accordingly, the image playback speed andquality of the image stream could be improved in real time.

On the other hand, if the ideal frame count of the second group ofpictures of the image stream is equal to 60 and the ideal frame count ofthe third group of pictures of the image stream is larger than 60 (e.g.90), meaning that variation in frame size is gradually decreasing (itrepresents that there are more still objects in the third group ofpictures than the second group of pictures), the processing unit 14 setsthe frame count of the third group of pictures as 90 and then generatesone initial frame and eighty-nine non-initial frames for the third groupof pictures (as shown in FIG. 4) after generating one initial frame andfifty-nine non-initial frames for the second group of pictures. Inanother embodiment, the processing unit 14 could directly set the framecount of the second group of pictures as 90 (as shown in FIG. 5), so asto reduce the overall data size of the image stream in real time forsaving the transmission time of the image stream.

To be noted, the aforesaid design could be further applied to thesubsequent groups of pictures in the image stream. That is, theprocessing unit 14 could obtain an ideal frame count of a subsequentgroup of pictures according to a ratio relationship of frame sizes of aninitial frame and at least one non-initial frame of a current group ofpictures in a predetermined frame segment (e.g. in 30 frames) or in apredetermined time segment (e.g. in 1s). Subsequently, the processingunit 14 could update the frame counts of the current and subsequentgroups of pictures according to a comparison result of the ideal framecounts of the current and subsequent groups of pictures, so as to ensurethat update of the frame counts of the current and subsequent groups ofpictures could be more smooth and could surely conform to the content ofthe image stream in real time. As for other related description, itcould be reasoned by analogy according to the aforesaid embodiments andomitted herein.

In summary, the present invention adopts the image processing method foroptimizing the frame count of the group of pictures according to thecontent of the image stream, so as to achieve the purpose that the groupof pictures could have a low frame count if variation in frame size islarge or could have a high frame count if variation in frame size issmall. In such a manner, the present invention could generate theinventive effect that the camera could transmit the image stream at theexpected compression efficiency as the frame count of the group ofpictures in the image stream is set to the ideal frame countcorresponding to variation in frame size.

Those skilled in the art will readily observe that numerousmodifications and alterations of the device and method may be made whileretaining the teachings of the invention. Accordingly, the abovedisclosure should be construed as limited only by the metes and boundsof the appended claims.

What is claimed is:
 1. An image processing method comprising: an imagecapturing unit receiving image data; outputting an image streamaccording to the image data, the image stream comprising an initialgroup of pictures and a next group of pictures, the initial group ofpictures comprising a first initial frame and at least one firstnon-initial frame, the next group of pictures comprising a secondinitial frame and at least one second non-initial frame; a processingunit calculating a first ideal frame count of the next group of picturesaccording to a formula: I _(GOP) =E(IP _(ratio)−1)/(1−E), 0<E<1, whereinI_(GOP) represents the first ideal frame count, E represents an expectedcompression efficiency of the image stream, and IP_(ratio) represents aframe ratio of a frame size of the first initial frame to an averageframe size of the at least one first non-initial frame of the initialgroup of pictures; the processing unit setting the first ideal framecount as a frame count of the next group of pictures; and a transmittingunit transmitting the image stream having the initial group of pictureswith the first initial frame and at least one first non-initial frameand the next group of pictures with the first ideal frame count.
 2. Theimage processing method of claim 1, wherein the image stream furthercomprises a further next group of pictures, the further next group ofpictures comprises a third initial frame and at least one thirdnon-initial frame, and the image processing method comprising: obtaininga second ideal frame count of the further next group of picturesaccording to a ratio relationship of frame sizes of the second initialframe and the at least one second non-initial frame of the next group ofpictures in the predetermined frame segment or the predetermined timesegment; wherein when the first ideal frame count is larger than thesecond ideal frame count, stop outputting the at least one secondnon-initial frame, set a frame count of the further next group ofpictures according to the second ideal frame count, and output the thirdinitial frame and the at least one third non-initial frame according tothe frame count of the further next group of pictures.
 3. The imageprocessing method of claim 2, wherein when the first ideal frame countis less than or equal to the second ideal frame count, the imageprocessing method further comprises: setting a frame count of the nextgroup of pictures according to the first ideal frame count; and settinga frame count of the further next group of pictures according to thesecond ideal frame count.
 4. The image processing method of claim 2,wherein when the first ideal frame count is less than or equal to thesecond ideal frame count, the image processing method further comprises:replacing the first ideal frame count with the second ideal frame count;and setting a frame count of the next group of pictures according to thesecond ideal frame count.
 5. The image processing method of claim 1,wherein when the first ideal frame count is within a frame segment, seta maximum value of the frame segment as the first ideal frame count. 6.The image processing method of claim 1 further comprising: obtaining thefirst ideal frame count according to an expected compression efficiencyand the ratio relationship.
 7. The image processing method of claim 1,wherein the image data is uncoded original image data.
 8. A cameracomprising: an image capturing unit for capturing images of an externalscene; a processing unit connected to the image capturing unit forreceiving an image data and generating an image stream corresponding tothe images of the external scene according to the image data, the imagestream comprising an initial group of pictures and a next group ofpictures, the initial group of pictures comprising a first initial frameand at least one first non-initial frame, the next group of picturescomprising a second initial frame and at least one second non-initialframe, the processing unit being further used for calculating a firstideal frame count of the next group of pictures and setting the firstideal frame count as a frame count of the next group of pictures, theprocessing unit calculating the first ideal frame count according to aformula: I _(GOP) =E(IP _(ratio)−1)/(1−E), 0<E<1, wherein I_(GOP)represents the first ideal frame count, E represents an expectedcompression efficiency of the image stream, and IP_(ratio) represents aframe ratio of a frame size of the first initial frame to an averageframe size of the at least one first non-initial frame of the initialgroup of pictures; and a transmitting unit connected to the processingunit for transmitting the image stream having the initial group ofpictures with the first initial frame and at least one first non-initialframe and the next group of pictures with the first ideal frame count.9. The camera of claim 8, wherein the image stream further comprises afurther next group of pictures, the further next group of picturescomprises a third initial frame and at least one third non-initialframe, and the processing unit is further used for obtaining a secondideal frame count of the further next group of pictures according to aratio relationship of frame sizes of the second initial frame and the atleast one second non-initial frame of the next group of pictures in thepredetermined frame segment or the predetermined time segment; whereinwhen the first ideal frame count is larger than the second ideal framecount, the processing unit stops outputting the at least one secondnon-initial frame, sets a frame count of the further next group ofpictures according to the second ideal frame count, and outputs thethird initial frame and the at least one third non-initial frameaccording to the frame count of the further next group of pictures. 10.The camera of claim 9, wherein when the first ideal frame count is lessthan or equal to the second ideal frame count, the processing unit setsa frame count of the next group of pictures according to the first idealframe count and sets a frame count of the further next group of picturesaccording to the second ideal frame count.
 11. The camera of claim 9,wherein when the first ideal frame count is less than or equal to thesecond ideal frame count, the processing unit replaces the first idealframe count with the second ideal frame count and sets a frame count ofthe next group of pictures according to the second ideal frame count.12. The camera of claim 8, wherein when the first ideal frame count iswithin a frame segment, the processing unit sets a maximum value of theframe segment as the first ideal frame count.
 13. The camera of claim 8,wherein the processing unit is further used for obtaining the firstideal frame count according to an expected compression efficiency andthe ratio relationship.